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1.0  INTRODUCTION 

We  report  here  on  the  development  and  testing  of  an 


automated  technique  for  classifying  clouds  in  the  visual  imagery 
from  an  orbiting  weather  satellite.  This  study  is  the  core  of  an 
effort  to  select  and  test  certain  algorithms  for  eventual  use  in 
automated  cloud  imagery  processing  at  Air  Force  Global  Weather 
Central  (AFGWC),  Omaha,  Nebraska. 

In  the  program  as  originally  conceived,  we  hoped  to 
exploit  the  highest  resolution  data,  fine  mode  at  0.3  nautical 
mile  (nm)  per  pixel,  from  both  visual  and  IR  sensors  on  board  the 
Defense  Meteorological  Satellite  Program  (DMSP)  Block  5D  satellites. 

The  approach  was  to  be  based  on  annularly  integrated  or  averaged 
Fourier  spectral  amplitudes,  termed  wave  number  spectra,  used 
as  input  features  to  a multivariate  normal  Bayesian  classifier. 

Similar  prior  studies  (Booth,  1973;  Sikula,  1974)  have  indicated 
potential  in  the  procedure.  For  a plethora  of  reasons  ranging 
from  satellite  instability,  to  AFGWC  work  priorities,  to  McIDAS 
hardware  failure,  no  Block  5D  data  were  processed  through  the 
classifier,  despite  the  best  efforts  of  all  involved.  Although 
Block  5C  data  were  processed  as  a backup,  they  did  not  include  IR 
imagery  at  adequate  resolution  for  classification.  Resolution  of 
the  available  IR  was  6 times  coarser  than  the  approximately  0.3  to 
0.7  nm  resolution  of  the  visual  imagery.  Thus,  while  conducting 
the  cloud  classification  study  with  visual  data  alone  provided 
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insight  and  experience  in  the  operation  of  a cloud  ciassit  er 
based  on  wave  number  spectral  components  as  features,  absent.  of 
the  richer  source  of  information  from  the  data  base  (IR  imagery, 
as  witness  Booth,  1973;  Koffler,  1973;  Lo  and  Johnson,  1971), 
was  a serious  limitation  to  classifier  performance. 

The  subsequent  body  of  tnis  report  consists  of  a brief 
historical  overview  of  the  present  project  and  prior  work  thr ’ 
led  up  to  it  (Section  2);  a description  of  the  . ' 

classifier  that  was  tested  here,  including  the  specific  si 
processinq  and  cl  ass  i f i ca t ion  algorithms  (Section  3);  a 
description  of  the  test  sample  of  cloud  images  and  the  assort' 
truth  data  (Section  4);  presentation  of  the  study  results 
(Section  5);  and  a discussion  of  the  significance  of  our  find  ~iqs 
for  further  development  and  study  of  automated  techniques  for 
cloud  classi f icat ion  (Section  6). 


2.0  HISTORICAL  OVERVIEW 

In  an  earlier  contract  effort  (Pickett  and  Blackman, 

1976),  it  was  determined  that  the  most  promising  next  step  for 
AFGWC  to  take  in  upgrading  its  capability  for  automated 
processing  of  weather  satellite  imagery  data  was  to  develop  a 
classifier  that  utilizes  spectral  features  of  the  data.  As  part 
of  that  initial  effort,  demonstrations  of  Fourier  spectral  analysis 
of  three  cloud  types  were  also  conducted  and  reviewed  (Blackman  and 
Pickett,  1977).  The  present  project  carried  the  process  forward. 
Algorithms  for  spectral  analysis  and  for  classification  based  on 
spectral  features,  were  selected  and  programmed,  with  most 
programming  done  by  AFGL  under  BBN  direction.  The  classifier 
comprised  of  these  algorithms  was  empirically  tested  on  a sample 
of  158  cloud  images  representing  six  different  categories  of 
clouds . 

The  original  aim  of  the  present  effort  was  to  develop  and 
test  the  automated  classifier  on  images  from  the  most  advanced 
series  of  satellites  now  serving  AFGWC,  the  DMSP  5D  series.  These 

5D  satellites  can  transmit  coincident  IR  and  visual  imagery  at  nomi  al 
spatial  resulution  of  0.3  nm.  Our  objectives  were  to  see  not 
only  how  good  a spectral  feature  classifier  might  be  for 
effective  and  efficient  digestion  of  data  at  that  high  a 
resolution,  but  also  to  determine  the  effectiveness  of  a spectral 
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classifier  usinq  visual  alone,  IR  data  alone  and  IR  and  visual  data 
in  combination. 

The  plan  was  to  do  initial  development  and  testing  of  t ' ? 
classifier  on  available  imagery  from  a DMSP  5C  satellite,  but  t.o 
switch  to  5D  data  when  it  became  available.  Repeated 
postponements  of  t.he  delivery  of  5D  data  led  to  adoption  of 
contingency  plans  for  debugging  and  testing  the  classifier  or  the 
be  data.  Plans  for  eventually  using  the  5D  data  were  several 
times  revised,  but  it  was  necessary  finally  to  abv.. 
altogether  for  utilizing  the  5D  data,  and  the  contract  eftoi 
eventually  focused  entirely  on  testing  and  analyzing  perfor.i  u 
of  the  classifier  with  5C  data.  Because  simultaneous  5C  I 
visual  VHR  images  at  equal  resolutions  are  not  available,  «. 
primary  impact  of  this  change  in  plans  with  respect  to  the  ° 
goals  was  that  it  limited  testing  the  classifier  to  just  visual 
data . 


To  ensure  continuity  of  the  present  effort  with  continuing 
related  work  at  AFGL,  the  contract  called  for  coding  of  the 
algorithms  by  AFGL  personnel  on  AFGL  computers.  Also,  the  truth 
data,  based  on  visual  classi f icat  ions  of  the  imagery  samples  as 
described  below,  had  to  be  accomplished  by  personnel  at  AFGL. 

The  success  of  this  overall  effort  in  spite  of  the  difficulties 
encountered  is  due  in  no  small  part  to  the  highly  responsive 
behavior  of  AFGL  personnel  in  adjusting  and  timing  their  support 
in  these  areas  to  suit  the  changing  study  plans  and  schedule. 
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3.0 


DESCRIPTION  OF  THE  AUTOMATED  CLASSIFIER 


Our  description  of  the  classifier  is  in  three  parts:  1) 
processing  of  the  raw  satellite  data  into  power  spectra;  2) 
conversion  of  the  power  spectra  into  average  wave  number  spectra; 
and  3)  computation  of  the  classification  statistics.  We  discuss 
each  step  briefly  here  and  refer  the  reader  to  section  3.2  of  our 
earlier  report  for  more  detail. 

3.1  Computation  of  Two-Dimensional  Power  Spectra 

The  Discrete  Fourier  Transform  (DFT)  in  two  dimensions  is 
defined  as 


- N M — i 2 11  [jf  (Ax)  + kf  (Ay)  1 

a (f  ,f  ) = E l a rj(Ax),  k(Ay)]  e x y 

x y -isi  v-i 

3 1 K 1 (i) 


with  a- *a  representing  Fourier  transform  pairs, 


If  we  let  N = M,  Ax  = Ay  = T/N  (with  T the  analysis  window 
size),  and  compute  fx,  fy  only  at  integer  multiples  of  T_i,  then 


- N N / . 

a (n,m)  = Z z a IT 

j=l  k-1  I N ' N 


-i2IT  | j n + kmj 


(2) 


which  is  a defining  form  for  the  FFT 


Computing  Equation  2 using  the  FFT  algorithm  product  an 
array  of  data  representing  the  first  quadrant  in  the  two 
dimensional  spectral  plane.  Because  of  the  spectrally  periodic 
nature  of  the  FFT,  the  entire  spectral  plane  can  be  generated 
f r om  first  quadrant  data,  allowing  direct  com put a t ion  of  wave 
number  spectra. 

3.2  Computation  of  Average  Wave  Number  Epee* ra 

Wave  number  spectral  components  are  computed  a. 
spectral  amplitudes  ( "afnfm)  ) for  those  elements  fall)  q win 
corresponding  annular  bands  in  the  frequency  plane.  Each  ! 
one  unit  in  width  and  symmetrically  disposed  radially  about 
integer  multiples  of  the  radial  “fundamental"  frequency  1/T. 
That  is,  at  least  for  n,m  <N/2,  compute 


where  [ ] extracts  the  qreatest  integer  not  exceeding  the 
expression  within,  and  add  a(n,m)  to  the  pth  wave  number.  + 
is  important  to  note,  however,  that  wave  numbers  computed  from 
FFT  indices  directly  are  not  referenced  to  the  principal  part  of 
the  (aliased)  spectrum  when  either  index  exceeds  the  Nyquist 
frequency. 

To  compute  wave  number  spectra  as  sums  over  annular 
regions,  we  subtract  any  given  index  from  N if  it  exceeds  N/2: 
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n ' =min ( n , N-l  ) 

(4) 

m'=min(m,N-m) 

(5) 

where  min(  ) extracts  the  smaller  of  the  two  values  within  the 
parentheses.  Equation  (3)  may  then  be  applied  to  n'  and  m'  to 
determine  the  proper  total  wave  number  spectrum.  Here,  we  are 
merely  exploiting  the  periodic  nature  of  the  FFT  and  representing 
n and  m by  their  values  within  the  principal  part  of  the 
spectrum.  Although  we  require  this  shifted  data  to  fall  within 
the  Nyquist  region  or  principal  spectral  part,  this  requirement 

does  not  simply  limit  the  useful  range  of  p to  p<N/2,  since  it  is 

/2 

clear  that  fx  = fy  = n/2  yields  p =-y-N.  In  essence,  aliasing 
occurs  above  frequencies  given  by  the  square  region  constraint 

( I I • I f y I £N/2 ) and  not  by  any  equivalent  one-dimensional 
constraint  on  p.  Thus,  we  ultimately  use  the  NxN  spectral 
components  of  the  first  quadrant,  i.e.,  all  the  FFT  data,  but 
combine  their  magnitudes  based  on  first  applying  Equations  (4) 
and  (5),  and  then  Equation  (3). 

To  qenerate  averaqe  wave  number  spectra,  each  wave  number 
spectral  component  is  divided  by  the  number  of  terms  that  went 
into  it.  The  components  are  scaled  such  that  the  zero  wave 
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number  component  »s  t Jin  average  value  of  the  spatial  dat*.  aiiay. 

3.1  Ouadrattc  Discriminant  Classification 

The  average  wave  number  spectra  ate  treated  as  input 
features  to  a Hayosian  quadiatic  discriminant  classitiei. 
Treating  the  class  conditional  probability  densities  foi  the 
teatuie  vectors  as  multivaiiate  normal,  we  have  (see  Duu.i  and 
Hart,  1973,  for  example) 

gjL(a)  ■ - | (a  - yi)t  l7*(a  - Mi)  - \ log  | T ^ | + log  Pf 


vU) 


Here,  a is  the  feature  vector  (wave  number  spectra),  "t"  de  ate- 
matrix  transpose,  gt(a)  the  discriminant  for  the  ith  class,  u 
the  mean  vector  for  that  class,  P(c  ) the  a pr lot  1 probability  et 
ith  class  occurrence,  and  ):  the  ith  covariance  matrix  with 

elements  given  by 


ik/i 


j/i ) (ak/ 


"k/i) 


(7) 


The  operator  El  I is  the  expectation  with  respect  to  the 
ensemble.  Expanding  Equation  (t>)  yields 


(?)  “ - \ ^ i'l'  a 4 (£“»  j..)1,!  + 0. 


(8) 


1 

. 

with 

°i  ” ' I H*  ,:i'  lij  - \ lo9  lEj  + log  P(0j)  (9| 

\ 

The  qeneral  form  of  Equation  (8)  indicates  quadratic  discriminant 
f unct ions . 

Our  concern  is  for  the  efficiency  and  hence  simplicity  of 

computat  ion.  Since  we  need  to  compute  0 j once  for  each  class  and 

- 1 * 

can  compute  once  and  store  ( > t u.)  na  a new  vector  u't  for  each 
class,  the  potential  computational  complexity  resides  in  the 

first  term  on  the  right  hand  side  of  Equation  8.  While  in 

2 

qeneral,  an  NxN  matrix  will  require  N'tN  multiplications  and  a 
similar  number  of  additions  (for  each  class),  we  make  a 
reasonable  assumption  >:  . * is  a diagonal  matrix  and  thus  we  can 
scale  each  element  of  i by  its  class  conditional  standard 
deviation  and  compute  the  norm  of  the  scaled  vector.  This  lattei 
computation  requires  only  ?N  multiplications  and  N additions  for 
each  class. 

In  the  cl  ass i f i ca t i on  experiment  as  conducted  with  visual 
data  from  Block  f>C,  the  covariance  matrix  was  est  imated  by  the 
sample  covariance  matrix,  with  off-diagonal  terms  set  to  zero. 

Had  we  been  able  to  process  Block  SD  data,  a somewhat  different 
covariance  matrix  estimator  would  have  been  used,  comprising  4 
submatrices,  each  with  nonzero  terms  only  on  its  main  diagonal. 
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The  composite  matrix  would  be  of  tri-diagonal  form,  with  * nzero 

terms  wherever  the  correlation  coefficient  involves  two  elei.  nts 

with  the  same  wave  number.  Also  considered  was  an  additional 

feature  for  classification,  the  cross-correlation  between  visual 

and  IR  data  at  zero  lag.  Thus,  for  N wave  numbers,  the  basic 

feature  vector  would  be  of  size  2N+1,  with  the  covariance  matrix 
2 

having  (2N+1)  elements,  of  which  4Ntl  are  nominally  nonzero. 
Because  of  the  larqe  number  of  elements  involved  ( ' is  53  in  the 
video  classification  experiment),  coalescence  of  wave 
spectra  would  have  been  performed. 
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THE  SAMPLE  OF  CLOUD  IMAGES 


This  description  is  divided  into  three  parts:  1) 
Specification  of  the  satellite  and  of  the  particular  orbits  and 
segments  of  orbits  from  which  the  images  were  taken;  2)  a 
general  description  of  the  test  images  and  the  procedures  for 
their  visual  classi f ieat  ion  and  selection;  and  3)  an  overview  of 
the  composition  of  the  sample. 

4.1  Satellite  and  Orbit  Specification 

The  cloud  images  were  recorded  by  DMSP  5C  vehicle  5528 
during  August,  1972.  The  image  samples  that  were  analyzed  were 
selected  from  very  high  resolution  (VHR)  visual  records  that  were 
displayed  on  a Man-Computer  Interactive  Data  Access  System  (MCIDAS) 
TV  screen.  Infrared  high  resolution  (HR)  images  of  the  same  area 
taken  at  the  same  time  were  used  to  assist  in  cloud  classification. 
Details  of  the  spatial  resolution  of  the  visual  and  IR  sensors  on  5C 
vehicles  are  given  in  Nichols,  et  al.  (1975,  pp.  186-187).  For  the 
central  2/3  of  the  satellite's  cross-track  scan,  to  which  selection 
of  visual  images  was  limited  in  this  study,  the  nominal  spatial 
resolution  for  VHR  visual  is  between  1/3  and  3/4  nm.  For  the  HR 
infrared  imagery,  corresponding  resolution  is  between  2 and  4 nm. 

Vehicle  5528  was  a noontime  satellite.  The  particular 
orbits  from  which  the  images  were  selected  were  numbers  5790  and 
6200.  Selection  was  from  that  part  of  each  orbit  where  the 
vehicle  was  passing  over  the  vicinity  of  Central  America. 


Table  i gives  the  general  location,  background  and  range  f cloud 
types  encountered. 

TABLE  1 

ORBITS  OF  VEHICLE  5528  FROM  WHICH 


IMAGES 

WERE  SELECTED 

ORBIT 

LATITUDE 

BACKGROUND 

CLOUDS 

5790 

NORTHERN 

LOW 

CARIBBEAN 
CENTRAL  AMERICA 
EASTERN  PACIFIC 

Ci  , Cb  , Cu , 
Cs 

6200 

NORTHERN 

LOW 

GULF  OF  MEXICO 
CARIBBEAN 

FLORIDA,  MEXICO, 
CUBA 

Ci , Co, 

Ac  , Sc 

4.2  Procedures  for  Image  Selection  and  Visual  Classifies  u. 

The  general  approach  was  to  have  meteorologists,  highly 
experienced  in  the  visual  interpretation  of  satellite  images, 
examine  full  resolution  TV  images  to  find,  classify,  and  excerpt 
small  square  regions  (75  x 75  pixels)  of  the  pictures  containing 
single  cloud  types,  to  serve  as  the  test  images.  Mixes  of  cloud 
such  as  might  occur  where  regions  of  different  cloud  types  at 
the  same  altitude  meet,  or  where  clouds  at  different  altitudes 
overlap,  were  strictly  avoided.  Mixed  (shore)  backgrounds  were 
also  avoided  as  much  as  possible,  and  excerpted  sample  images 
were  not  allowed  to  overlap. 

In  the  first  stage  of  the  visual  search  for  the  test 
images,  large  scale  views  of  both  visual  and  IR  imagery  were  examined. 
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For  this  stage,  the  visual  data  were  electronically  degraded  to 
match  the  lower  resolution  if  the  1R  imagery.  These  paired  large- 
scale  views  provided  a basis  for  the  observer  to  orient  himselt 
to  the  general  geographic  location,  the  background  against  which 
the  clouds  had  to  be  interpreted  and  recognized  (land,  water, 
shore),  and  general  features  of  the  cloud  distribution  at 
mesoscale.  The  IR  data  were  important  in  helping  to  separate 
low  and  high  clouds.  Final  distinctions  among  cloud  types  at  a 
given  altitude  were  made  on  the  basis  of  visual  inspection  of 
full  resolution  views  of  the  V11R  visual  data  for  each  candidate  image. 


4.3  Composition  of  the  Image  Sample 


An  attempt  was  made  to  find  test  images  in  each  of  the 
following  eight  categories:  cumulus  (Cu) , stratocumulus  (Sc), 
stratus  (St),  altocumulus  (At),  altostratus/n imbost rat  us  (As  Ns) , 
cumulonimbus  (Cb) , cirrus  (Ci),  and  cirrostratus  (Cs) . Several 
samples  of  clear  sky  were  also  sought.  In  all,  168  cloud  images 
and  14  clear-sky  samples  were  selected. 


Figure  1 shows  the  composition  of  the  image  sample.  Most 
numerous  (66)  are  images  of  cumulus  clouds.  Middle-level  clouds 
(As/Ns  and  Ac)  all  by  themselves  did  not  occur  with  any  appreciable 
frequency.  No  As/Ns  or  St  and  just  2 Ac  samples  were  found. 
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RESULTS  OF  THE  STUDY 


5.J0 


We  present  first  what  we  found  reqardinq  the  spectral 
characteristics  of  the  various  cloud  types  as  a result  of  just 
the  signal  processing.  We  then  examine  tin*  performance  of 
the  classifier,  how  accurately  it  classifies,  what  confusions  it 
makes  among  the  clouds  when  it  does  err,  and  how  that  pattern  ot 
confusions  may  be  taken  into  account  in  estimating  accuracy  of 
classification  were  the  cloud  types  optimally  partitioned. 

Finally,  we  report  on  the  procedure  and  results  of  a 
multidimensional  scaling  analysis  of  the  error  data  which 
provides  insight  into  the  number  of  orthogonal  dimensions  used 
effectively  by  the  classifier. 

5.1  Spectral  Characteristics  of  the  Cloud  Types 

The  averaqe  wave  amplitudes  for  each  of  the  six  cloud 
types  found  is  given  in  Figure  2.  The  discriminating  information 
in  this  family  of  curves  appears  to  be  mainly  in  the  d.c.  (zero 
wave  number)  components  and  in  one  or  two  properties  of  overall 
shape,  e.g.,  bandwidth  and/or  amplitude  at  some  representative 
mid  range  wave  number.  Not  surprisingly,  Cb  has  by  far  the  largest 
d.c.  component  and  Cu  has  the  greatest  bandwidth.  The  clear 
category  is,  appropriately,  the  lowest  in  both  of  these  aspects. 

It  is  important,  however,  to  temper  interpretation  of  the  apparent 
orderliness  of  these  data  with  a consideration  of  variability  of 
these  functions,  smooth  as  they  are,  among  realizations  within 
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AVERAGE  WAVE  AMPLITUDE 


is  illustrated  in 


categories.  The  variability  is  substantial,  <ts 
Figure  3,  which  shows  the  six  realizations  for  the  Cs  category. 

Breaks  in  the  average  wave  amplitude  spectral  plots  of 
Figures  2 and  3,  occurring  between  wave  numbers  37  and  38,  have 
been  inserted  for  clarity.  The  abrupt  change  in  level  for  many 
but  not  all  of  the  curves  as  they  transit  this  region,  is 
believed  due  to  the  combined  effects  of  aliasing  and  minor 
pattern  anisotropy.  As  stated  above,  the  Nyquist  frequency  is  a 
function  of  orientation  for  two-dimensional  FFTs . Here,  it  is 
75/2  = 37.5  in  vertical  and  horizontal  directions,  and  increases 
to  75/  / 2 - 53  at  11/4  multiples  that  are  not  multiples  of  H/2.  If 
a cloud  pattern  is  anisotropic  such  that  its  spectrum  extends 
further  in  the  vertical  or  horizontal  than  at  any  diagonal 
azimuth,  aliasinq  will  be  more  severe  than  if  the  pattern  were 
aligned  with  its  maximum  spectral  extent  coincident  with  the 
direction  of  minimal  aliasing  (maximal  Nyquist).  The  impact  of 
anisotropy  and  its  interaction  with  aliasing  on  the  performance 
of  the  classifier  is  unknown.  Note,  however,  that  the 
sem i- loga r i thm ic  nature  of  the  plots  greatly  exaggerates  the 
ef  feet . 
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AVERAGE  WAVE  AMPLITUDE 


5.2 


Accuracy  of  Automate)  Cl  ass  i f ic.it  i on 


The  accuracy  of  classificat  ion  is  derive)  I rom  the  data 
given  in  Figure  A in  the  form  of  two  matrices,  which  relate  true 
(visual)  classification  to  automated  classification.  The  matrix 
on  the  left  gives  the  data  in  frequency  form.  On  the  right  they 
appear  in  probability  form,  which  is  the  frequency  form 
normalized  to  unity.  The  columns  of  each  matrix  stand  for  the 
true  (visual)  classifications.  The  rows  stand  for  the 
classification  as  issued  by  the  automated  classifier.  The  main 
diagonal  cells  (those  runninq  from  the  upper  left  to  the  lower 
right  corner)  show  in  the  left  (right)  matrix  tne  frequencies 
(probabilities)  of  correct  classi f icat  ion , i.e.,  the  number 
(relative  number)  of  samples  of  each  cloud  type  that  were 
correctly  classified. 

Overall  accuracy  (probability  of  correct  c 1 ass i f i ca t i on ) 
is  determined  by  summing  over  the  frequencies  in  the  diagonal 
cells  and  dividing  by  the  total  number  of  samples.  Thus  the 
probability  of  a correct  classification  is  72/158  = 0.46. 

Accuracy  of  classification  within  each  of  the  cloud  types 
is  also  of  interest.  The  cloud  type  with  the  highest  probability 
of  correct  classification  is  Cs , with  a probability  of  0.83. 
Probability  of  correct  classi f icat  ion  is  no  less  than  0.38  for 
any  of  the  six  categories.  Of  course,  it  is  important  to  observe 
that  the  confidence  bounds  for  these  probabilities  vary 
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Figure  4.  Matrices  shov  ing  accuracy  of  class:  'ica  , frequency  data 
on  the  left  and  probability  data  or.  thr  nt. 


■■■ 


considerably  from  one  class  to  another  as  a function  of  the 
number  of  samples  in  each  category.  Performance  in  category  Ac, 
where  p = 0.5,  is  not  well  estimated,  since  it  is  based  on  only 
two  samples.  A similar  small  sample  disclaimer  can  be  made  for 
Cs , where  p = 0.83,  since  it  is  based  on  only  six  samples. 

Accuracy  of  classification  on  the  sample  as  a whole  has  to 
be  gauged  against  chance  performance  following  one  or  another 
strategy  by  a "blind  observer".  If  the  blind  observer  works 
without  knowledge  of  the  a priori  probabilities,  and  knows  only 
what  the  six  possible  categories  are,  all  strategies  are 
equivalent.  Chance  performance  yields  on  the  average 
p = 1/6  = 0.17.  If  the  blind  observer  knows  the  a priori 
probabilities,  then  the  best  strategy  is  to  classify  every  sample 
as  a member  of  the  most  probable  category,  which,  in  this 
situation,  is  Cu  with  66  out  of  the  158  samples.  That  strategy 
would  allow  the  blind  observer  to  attain  p = 66/158  = 0.42. 

Since  the  automated  classifier  is  Bayesian,  i.e.,  uses  a 
priori  probabilities,  it  is  appropriate  to  compare  its 
performance  to  the  performance  of  the  blind  observer  using  the  a 
priori  probabilities.  Thus,  the  unconditional  comparison  of  the 
automated  classifier  to  chance  classification  is  0.46  to  0.42. 

On  the  face,  the  classifier  performance  is  not  very  impressive, 
but  there  is  considerably  more  to  be  said.  First,  it  is  clear 
from  the  pattern  of  classifications,  that  the  automatic 
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classifier  is  not  leaning  heavily  on  the  a priori  probabi  ties. 
Note  that  it  assigns  only  39  samples  to  the  Cu  category.  Ai  ;o , 
of  particular  significance  is  the  pattern  of  confusions  when  th. 
automated  classifier  errs,  and  how  its  performance  improves  when 
certain  cloud  categories  are  combined.  Finally,  we  must  consider 
the  class  conditional  performance  taken  as  a whole  and  compared 
with  the  class  conditional  performance  of  the  blind  observer. 

For  Cu,  of  course,  the  blind  observer  is  perfectly  . -c  irate, 
whereas  for  any  other  class,  he  is  always  wrong.  In  v. 
except  for  Cs  which  is  a small  class  with  6 occurrences, 
performance  of  the  automated  classifier  is  closely 
class- i ndependent  as  regards  accuracy.  While  the  accuracy  . lie 
automated  visual  c 1 ass  i f i e r leaves  something  to  be  desired  (' 
data,  perhaps),  its  performance  is  consistent  with  the  1,0  cus>. 
function  implicitly  used  (all  errors  equally  bad,  all  correct 
answers  equally  qood) . 

5.3  Information  in  the  Pattern  of  Confusions:  Partitioning 

The  off-diaqonal  cells  in  the  matrices  of  Figure  4 show 
how  the  classifier  performs  when  it  errs.  The  pattern  of  errors 
is  clearly  not  random.  Certain  cloud  types  are  more  probably 
confused  than  others.  Note  that  the  errors  tend  to  cluster  in 
the  upper  left  and  lower  right  quadrants.  Thus,  for  the  super 
categories  ( ScCuAc  Versus  CiCsCb),  performance  is  rather  highly 
accurate.  The  automated  classifier  calls  a sample  from  ScCuAc 
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either  Sc,  Cu,  or  Ac  with  p = 68/89  = 0.74.  Similarly,  it  calls 
a sample  from  the  CiCsCb  either  Ci  , Cs  or  Cb  with 
p = 61/69  = 0.88.  The  overall  accuracy  of  the  automated 
classifier  with  respect  to  these  super  categories  is  thus, 
p = 129/158  = 0.82. 

This  level  of  accuracy,  of  course,  also  has  to  he  gauged 
relative  to  chance,  the  performance  of  a blind  observer  which 
assigns  every  sample  to  the  super  cateqory  with  the  larger  a 
Pfjotj  probability,  which  in  this  case  is  ScCuAc,  with  its  89 
samples  out  of  the  total  of  158  (p  = 0.56).  Thus,  the  compar ison 
of  the  automated  classifier  to  chance  classification,  taking  the 
pattern  of  confusions  into  account  in  this  particular  way,  is 
0.82  to  0.56.  Now  wo  can  see  a substantial  advantage  of  the 
classifier  over  chance.  Of  course,  we  have  to  acknowledge  that 
this  particular  pattern  of  confusions  would  itself  be  subject  to 
variation  over  repeated  tests,  and  the  exact  nature  of  the 
pattern  of  confusions  and  how  best  to  partition  the  sample  so  as 
to  optimize  performance  of  the  classifiet  would  depend  on 
obtaining  more  reliable  estimates  of  the  confusion  probabilities. 
We  can  only  illustrate  here  the  possible  gains  from  such 
optimization. 
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‘'•4  Information  in  t ho  Pattern  of  Confusions:  Dimonsloi  lity 

Tin*  automated  e I ass  i f i et  locates  each  sample  i n *>  t spaio, 
•mi'Ii  ilinu'nsion  ot  t li«'  spare  oot  r espond  i iuj  to  .1  wave  number  . 

Hypo  1 t|iiad r i os  partition  tin*  spare  into  mutually  exclusive  icijion 
cot  1 espond  i nq  to  t'.u'h  ot  t lie  six  rloud  types.  Samples,  wliioli  h^ 
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These  measures  of  similarity  are  input  to  a computer 
program  (see  Takane  et  al.  (1977)  and  specifications  in  Appendix 
A of  this  report])  for  nonmetric  multidimensional  scaling.  That 
proqram  treats  these  inputs  as  measures  of  the  distance,  in 
n-space,  among  the  categories.  It  finds,  for  selectable  values 
of  n,  that  positioning  of  the  six  categories  which  best  accounts 
for  the  distances  in  terms  of  similarity,  and  it  measures  how 
much  of  the  variance  of  these  distances  is  accounted  for  by  the 
model.  For  n = 6,  by  definition,  1 0 M % of  the  variance  can  be 
accounted  for.  Is  there  a dimensionality  less  than  6 which 
accounts  for  essentially  all  of  t he  variance? 

What  we  find  in  this  case  is  that  only  1 dimensions  are 
needed  to  account  for  over  98%  of  the  variance.  Thus,  though  the 
classifier  nominally  utilizes  53  dimensions  in  classifying  the 
stimuli,  the  pattern  of  confusions  suggests  that  the  effective 
number  of  orthogonal  dimensions  being  utilized  (not  necessarily 
one  to  one  with  any  of  the  original  wave  number  inputs)  is  no 
more  than  three.  Each  of  those  three  dimensions  could  of  course 
be  derived  from  some  combination  of  the  wave  number  inputs,  and 
we  have  no  way  of  telling  what  those  derivations  are  without 
conducting  further  studies  beyond  the  scope  of  the  present 
contract.  However,  we  can  look  to  see  if  there  is  any  obvious 
relationship  between  any  one  of  the  three  dimensions  in  the 
confusion  model  and  observable  dimensions  of  differences  among 
the  six  average  spectral  functions  shown  in  Kigute  2.  We  have 
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alreadV  observed  that  they  tend  to  differ  in  terms  of  tin  .c. 
component  and  in  terms  of  a few  qeneral  features  of  shape  re  ited 
to  in  id  range  level  or  bandwidth.  Could  these  features  correspor,  1 
to  anv  of  the  dimensions  in  the  confusion  model? 

from  Figure  ? we  see  that  with  respect  to  magnitude  of  the 
d.c.  component  (height  at  wavt  number  tt ) the  cloud  types  order 
from  low  to  high:  Ac,  Sc,  Cu,  Ci , Cs  and  cp.  Taking  height  or 
t lie  curves  at  wave  number  35  as  our  point  of  re:  --  > t f i > ; 

that  the  order  of  cloud  categories  from  low  to  high  is:  C. 

Ac,  Sc,  Cb,  and  Cu . 

Our  inspection  of  the  positioning  of  the  cloud  types  i , 
the  confusion  model  showed  that  the  order  of  stimuli  in  dimet  t,.n 
1 of  the  model  is  almost  identical  to  their  order  with  respect  u. 
magnitude  of  the  d.c.  component;  and  that  their  order  on  dimension 
3 is  almost  identical  to  their  order  at  the  representative  high  mic 
range  wave  number.  The  plot  of  the  cloud  types  on  dimensions  1 
and  3 of  the  confusion  model  is  given  in  Figure  5,  and  the  rt 
can  satisfy  himself  regarding  these  assertions. 
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Fiqure  5.  Plot  showing  locations  of  each  cloud 
type  in  two  of  the  three  dimensions 
of  the  scaling  model. 
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6.B  INTERPRETATIONS,  CONCLUSIONS  AND  RECOMMENDATIONS 

In  this  concluding  section  of  the  report,  we  attempt  tv 
place  the  present  study  in  its  proper  context.  The  original  aim 
was  to  develop  and  test  a cloud  classifier  based  on  wave  number 
spectra  as  features,  jointly  expl o i t i ng  visual  and  1R  fine  mode 
data  "saves"  from  Block  5D  satellites.  We  have,  as  discussed  and 
for  the  reasons  stated  above,  developed  and  tested  a ("loud 
classifier  based  on  block  5C  visual  data  alone. 

6.1  Interpretations 

Given  that  various  authors  have  found  visual  data  in  th 
absence  of  IR  data  to  be  without  much  virtue  vis  a vis  automate 
Bayesian  classification  (e.g.,  Booth,  1973;  and  Parikh,  1977, 

1978  who  claims  no  significant  value  in  textural  discriminators 
for  cloud  typing),  the  46%  classification  accuracy  found  here  is 
hardly  surprising.  While  Booth  quotes,  for  the  dependent  data 
set,  c l ass i f i ca t i on  accuracies  of  76%  and  81%  for  the  respect ivc 
6-category  and  S-cateqory  scenarios,  these  results: 

1)  use  bivariate  data  (visual  and  IR) 

2)  include  clear  sky  as  a category. 

Table  2 shows  the  results  from  Booth's  classifier,  when 
classi f icat ion  is  based  on  visual  data  alone. 
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TARLE  2 

BOOTH'S  CLASSIFIER  BASED  ON  VISUAL  ALONE 


6 

-Category 

5 -C  a t eg  o r y 

Type 

# 

cases 

% correct 

Type 

t cases  % correct 

Clear 

113 

75 

Clear 

1 1 3 

6 5 

Cu 

258 

60 

Cu/Sc 

413 

59 

Sc 

155 

35 

Cb 

117 

63 

Cb 

117 

65 

Ci 

174 

55 

Ci 

174 

46 

Ci+low 

233 

46 

C i + 1 ow 

233 

38 

Totals: 

1050 

54 

10  50 

54 

No  Clear: 

937 

52 

937 

52 

Note  that  the 

6-  and  5 

-categor y 

class i f i e r s 

each  operates 

at 

roughly  54% 

accuracy , 

or  at  52% 

if  the  clear 

category  is 

artificially  expunged  from  consideration.  For  comparison, 

Table  3 shows  results  from  the  wave  number  spectral  classifier  we 
have  utilized,  with  6-category,  quasi  5-category,  and  quasi 
4-cateqory  variations  on  the  basic  theme. 


TABLE  3 

WAVE  NUMBER  SPECTRAL  CLASSIFIER 


6-Category  Quasi  5-Category  Quasi  4-Category 


Type 

teases 

%cor . 

Type 

teases 

%cor . 

Type 

teases 

%cor 

Sc 

21 

48 

Sc 

21 

48 

Sc/Cu 

87 

76 

Cu 

66 

48 

Cu 

66 

48 

Ac 

2 

50 

Ac 

2 

50 

Ac 

2 

50 

Ci 

34 

38 

Ci 

34 

38 

Ci 

34 

3 8 

Cb 

29 

38 

Cb/Cs 

35 

69 

Cb/Cs 

35 

69 

Cs 

6 

83 

Totals:  158 

46 

158 

51 

158 

66 

Here,  note  that  our  6 categories  are  all  pure  cloud  types.  The 
quasi-5  classifier  refers  to  formally  combining  Cs  and  Ch  into 
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one  class,  thereby  representing  a realizable  though  subop  Imal 
approximation  to  a 5-category  system.  Comparing  the  quasi- 
classifier to  Booth's  "6-category-minus-clear"  results  shows 
close  parity,  i.e.,  51%  to  52%.  Combining  Cu  and  Sc 
classifications  produces  the  quasi-4  classifier.  Comparing  these 
results  (66%  correct)  to  Booth's  5-ca t egory-m i nus-c 1 ea r (52% 
correct)  shows  the  visual  wave  number  spectral  classifier  ir  a 
better  light.  Taking  all  the  above  factors  and  ci'nipir1’  sons  into 
account,  we  view  the  present  experiment  as  being  con~- 
prior  results  and  with  our  reasonable  expectations,  given  the 
lack  of  IR  data. 

6.2  Recommendations 

It  is  difficult  to  avoid  the  temptation  to  recommend  tna 
the  oriqinal  question  of  bispectral  classifier  performance  be 
answered,  when  Block  5D  data  and  the  facilities  to  process  the 
data  are  simultaneously  available.  Recent  studies  (Parikh,  1977 
and  1978),  suggesting  that  textural  features  are  of  little  val  . e 
for  cloud  classification,  are  not.  hiqhly  relevant  to  the  success 
of  classifying  Block  5D  imagery  via  wave  number  spectra  because 
these  studies  used  low  resolution  data  and  did  not  use  wave 
number  spectral  components  as  textural  features.  Certainly, 
Parikh’s  classifier  accuracies  using  only  first  order  visual  and 
IR  statistical  measures  as  features  are  impressive,  but  the 
number  of  cloud  classes  (3  or  4)  and  the  class  definitions  were 
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not  overly  ambitious.  There  is  an  irony  in  Parikh's  findings  that 
texture  is  not  important  for  automated  classifiers,  despite  the 
central  importance  of  texture  to  the  weather  expert  who  is 
providing  the  truth  against  which  the  classifiers  are  judged. 

6. 3 Conclusions 

One  small  and  preliminary  experiment  does  not  motivate  the 
drawing  of  many  profound  conclusions.  We  have  found  strong 
morphological  similarities  in  the  wave  number  spectra  of  all 
cloud  classes,  but  also  quantitative  differences  sufficient  for 
moderate  classifier  performance.  Clearly,  use  of  IR  wave  number 
spectra  is  expected  to  improve  classifier  performance,  perhaps  by 
some  20  to  30%  as  in  Booth's  study.  The  potential  impact  of  the 
IR-visual  cross  cor r e la t ion  feature  cannot  easily  be  assessed,  as 
we  have  been  unable  to  find  prior  studies  in  which  this  obvious 
feature  was  used. 
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